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were those of the Illinois Certification Testing System Surveys, 
which were sent to over 13,000 public school educators in 1986 (827. 
response rate) and approximately 9,000 educators in 1993 (77% 
response rate). The job-analysis results, contrary to expectation, 
suggest that the body of knowledge that classroom teachers deem 
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as rapidly as might be expected, and that teachers may not change 
their priorities and incorporate new recommendations promptly. The 
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Introduction 

During the past decade there has been a significant amount of activity occurring in the public 
schools in terms of reexamining the goals, content, and instructional methodology of the 
curriculum of the primary academic subject-matter areas. Virtually no content discipline has 
ignored the call to examine and revise the focus of public school instruction. The results of 
these reviews, that have typically been directed by national and state curriculum organizations, 
have led to comprehensive changes in the way that educators believe that the various subject 
areas should be organized and taught (Lioyd-Jones and Lunsford, 1989; Elbow, 1990; National 
Council of Teachers of Mathematics, 1989; Rutherford and Ahlgren, 1989; Seixas, 1993). 
Given these curriculum reforms, one might expect that public school educators would have 
different perspectives on the importance of various elements of the curriculum in the eariy 
1990s as opposed to their perspectives in the mid-1980s. In thk investigation, public school 
ratings of the importance subject-matter test objectives associated with teacher certification 
tests were examined by means by means of a job analysis survey conducted in 1986 and from a 
similar survey conducted in 1993. It was predicted that there would be a high degree of 
change in the importance ratings assigned to these test objectives reflecting the curriculum 
changes that had taken place over this seven year period. 

The Illinois Certification Testing System (ICTS) 

In 1986, the Illinois State Board of Education (ISBE), as part of a legislative initiative to 
improve educational quality, began with the development of 53 teacher, administrator, and 
pupil personnel service certification tests and a test of basic skills. Throughout the test 
development process, continuous efforts were made to involve Illinois public school educators 
and teacher educators in test domain and objective development, job analysis, item review, 
content validation, and standard setting activities. 

As part of initial test development in 1986, a set of test objectives for each test field that had 
been previously reviewed by committees of Illinois educators, were distributed to thousands of 
additional Illinois public school educators and college faculty involved in teacher preparation 
(Silvestro, Clayton, and Glenn, 1989). In 1993, the ISBE decided that as part of the test 
updating process for each test field, a job analysis survey would be conducted for all test 
objectives, similar to the job analysis survey that was conducted in 1986. The data from this 
urvey could then be used to determine the degree to which existing test objectives would need 
io be modified. 



Job Analysis Survey Methodology 

While the job analysis surveys that were developed and administered in 1986 and in 1993 were 
distributed to Illinois public school educators and to college faculty involved in the preparation 
of teaching certification candidates, for the current investigation, the focus will be on the 
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survey methodology used for public school educators. The survey methodology was identical 
for 1986 and 1993. 

Job analysis survey form. The first section of the survey form requested educator 
demographic information, including teaching certification(s) held, level of education 
completed, ethnic and racial background, years of professional experience, and the grade 
level (s) at which they were currently working. 

The second section of the survey form requested educators to rate the importance of all of the 
40 to 60 content knowledge objectives for their field. These objectives had been previously 
reviewed, revised, and approved *>y committees of Illinois public school educators and teacher 
educators. The actual wording of the importance rating stimulus for each test objective was as 
follows: 

IMPORTANCE: Rate the importance of the content of this objective to the job of an 
educator in this field in Illinois. 

1 = no importance 

2 = little importance 

3 = moderate importance 

4 = great importance 

5 = very great importance 

In addition to the objective importance rating, public school educators were also asked to 
indicate the primary reason why they assigned a "no importance" rating to any objective. They 
were also asked to indicate the amount of time they spent teaching or using the content of each 
objective during the school year. 

Distribution of surveys. Several steps were used in the identification and selection of job 
analysis survey participants. For public school educators, a data tape was provided by the 
Illinois State Board of Education. The data tape was derived from the ISBE Teacher Service 
Record, which listed all certified and practicing public school educators in each certification 
field. From this listing, a stratified random sample of educators was selected for the job 
analysis survey for each field. Survey materials were provided to school district 
superintendents, or building principals (Chicago only), who in turn distributed the materials to 
selected public school educators. Surveys were sent to over 13,000 public school educators in 
1986 and to approximately 9,000 educators in 1993. 

Survey response rates. The overall public school educator response rates for the job analysis 
survey was 82 percent for the 1986 survey and 77 percent for 1993. 
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Results 

While the job analysis surveys were conducted for all 53 ICTS subject-matter fields, the 
following fields were selected for specific study of the consistency of objective importance 
ratings by public school educators from 1936 to 1993. 



Elementary 
Learning Disabilities 
Guidance 

General Administration 
History 



English 

Mathematics 

General Science 

Music (K-12) 

Physical Education (K-12) 



These fields were selected for two important reasons. First, they represent teacher certification 
fields with large populations of educators in Illinois. Second, they represent a broad spectrum 
of disciplines including elementary education, special education, the humanities, sciences, pupil 
personnel services, and school administration. 

For the mean objective importance ratings for each objective for each of the 10 fields studied, 
Wests were computed. The following table summarizes significant objective importance 
ratings changes by test field from 1986 to 1993. 



FIELD 


N_OF OBJECTIVES 
RATED 


M OF SIGNIFICANT OBJ. 
RATING CHANGES 


Elementary 


SI 


7 


Learning Disabilities 


42 


2 


Guidance 


53 


6 


General Administrative 


40 


5 


History 


56 


3 


English 


46 


8 


Mathematics 


48 


12 


General Science 


56 


1 


Music (K-12) 


42 


2 


Physical Education (K-12) 


48 


19 



Overall, there was a high degree of consistency in th*, importance ratings assigned to the test 
objectives by public school educators across the 10 test fields. With the exception of 
Mathematics (25 percent of objective ratings changed) and Physical Education (40 percent), the 
remaining fields had minimal changes in objective importance ratings over the seven year 
period between the two job analysis surveys. Where tf»? objective ratings did change 
significantly, 37 of the objectives were rated higher v) 1993 and 28 were rated higher in 1986. 
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In examining specific objective importance ratings changes by test field, there were no clear 
patterns in terms of the changes being confined to particular test subareas or content knowledge 
domains for eight of the 10 fields. For Mathematics, four objectives from the test subarea 
"Basic Mathematical Principles and Applications" were given higher importance ratings in 
1993. For Physical Education, two test subareas had a number of objectives with importance 
ratings changes. For the test subarea "Physical Fitness", six of the seven test objectives were 
rated higher in 1993 than in 1986. For the test subarea "Motor Skill Acquisition", nine of 21 
test objectives were rated higher in 1993 than in 1986. 



Discussion 

The results of this study, while contrary to expectations, are important for a number of 
reasons. First, the study suggests that the body of knowledge that classroom teachers deem to 
be important for teachers to know In order to perform effectively on the job, may not change 
as rapidly as oi.e might expect from reading the literature on changes in the public school 
curriculum. Second, the study suggests that there may be, within each subject-matter field, 
core knowledge that educators hold to be valuable for student to learn, even in the face of 
continually evolving instructional methodology that is characteristic of each subject-matter 
discipline. Since the ICTS program does focus on the knowledge needed by entry-level 
educators to perform on the job, the test objectives are likely to reflect the core knowledge. 
Third, from a somewhat negative perspective, the results of this study may suggest that 
educators are reluctant to modify their content knowledge priorities for students and to 
incorporate into their teaching, new priorities suggested by national curriculum associations 
and other agents of educational change. 

The Illinois State Board of Education has embarked on an extensive program of ICTS test 
review and updating. It will be interesting to note the degree to which content advisory 
committees, which include Illinois teachers and teacher educators for each subject-matter field, 
seek to make changes in the objectives that currently serve as the basis of each assessment. 
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